8.1.1 Invisible, Inaccessible or Unrelatable Data

In pursuit of visible, understandable data [6.1.1; 6.1.2], the first obstacle encountered is that most personal data is invisible, inaccessible or unrelatable. It is trapped in service providers’ databases, or on different devices or hard drives, or by software limitations and proprietary file formats (Abiteboul, André and Kaplan, 2015; Bowyer, 2018). My research participants spoke of ‘not knowing’ what data exists and of being ‘in the dark’. Case Study Two showed that even where data is accessible, it is not relatable (legible (Mortier et al., 2014); 2.3.2). The objective here, addressed in Insight 1, is to ensure that people not only have awareness of their data, but can understand (‘make sense’ [Gurstein (2011); 2.1.4]) of what it means.

To make data meaningful, it needs to be expressed as information about your life.

8.1.2 The Personal Data Diaspora

Another important obstacle to consider here is what I call the Personal Data Diaspora¹. As illustrated by Imogen Heap’s quote opening Chapter 1, an individual’s personal data is typically very widely dispersed, and there is no central, holistic view of one’s data. For example, if I consider just my movement tracking data, I have over time accumulated activity logs from walking, running, cycling, and driving which are stored by Nike+, MyFitnessPal, Strava, Google Fit, Fitbit, Apple Health and Google Maps, not to mention the records remaining on my different smart watches, smartphones and hard drives. This is the SI problem of Integration (Li, Forlizzi and Dey, 2010) [2.2.3]. EAs well as the challenge of managing one’s data ecosystem [2.2.4], this makes impossible to view physical activity history in one place, to spot patterns over time or make comparisons. To overcome this, we need interfaces that recognise the scattered reality of each individual’s personal data, and begin to make that ecosystem visible and understandable [see 8.3 and 8.4 below].

Data awareness and understanding is a problem of representation. Invisible data should be visibly represented. All data should be represented as contextually relatable life information.

8.2.1 Immobile, Inaccessible or Unmalleable Data

To improve the useability of data, let us consider what properties of data make it hard to use. Most personal data is immobile, inacessible, unmalleable and not interrogable.

It is immobile in that is very difficult to move a dataset out of its environment. Most data exists in organisations’ internal databases, where it is tightly coupled to technology stacks, interfaces and business processes. Separating one’s data from the service that holds it is difficult and often impossible. It is inaccessible to individuals (in the sense of effective access (Gurstein, 2011)). Data access requests such as GDPR are typically satisfied by creating a copy of the data, creating problems of delay, divergence and understanding. Even then, returned data is incomplete [5.4.2]. Its accessibility is also hindered by the technical nature of data. Data is often be stored in complex proprietary structures which are designed for the algorithmic efficiency of the specific business operations rather than for general-purpose re-use.

People need to be able to ask questions of their data [Table 5.4; 4.3.2]. But data is not interrogable. It must stand for itself, yet there is no obvious way to ask a question about the meaning of the data or its ability to answer a particular question. To ask questions of data requires either the co-operation of the data holder or advanced technical skills in data querying and analysis. Data needs to be malleable—capable of being broken down, looked at from different perspectives, and reconstituted in different ways. This goes beyond visually representing the data, and implies an ability to interact with the data to produce new interpretations and insights to investigate specific questions.

To overcome these obstacles, data must be freed from its current constraints and moved into environments where it can be freely examined and reconstituted without restriction. This leads to Insight 2:

‘All related items should be classified together regardless of technological format’—Bergman, Beyth-Marom and Nachmias (2003)

• creating,
• deleting,
• moving,
• grouping,
• annotating,
• copying,
• sharing,
• modifying,
• labelling,
• organising, and
• separating.

“Designers are in general not trained and prepared to work with data. They’re not equipped with the right tools. Data manipulation is not part of the schools’ curriculum and designers are rarely interested in understanding data.”—(Deckers, 2018).

As Insight 3 show, data will only become useable once we change its nature. Since the 1970s, drawing on the then-common metaphor of a filing cabinet, computers have considered files as the basic material that users will interact with. Where we do interact with data as information instead of files, that information is typically presented in limited contexts within certain products or apps [Insight 1]. To move up the DIKW pyramid [2.1], we need smarter computer systems, that move beyond files (Bowyer, 2011). We need systems whose basic material is not files, but pieces of human information.

We need a human information operating system.

8.3.1 Complex and Invisible Personal Data Ecosystems

As established [2.2.5; 2.3; 6.2; 7.2], HDR cannot be made effective without a sea change in the way that individuals are able to interact with the complex ecosystem of personal data they each inhabit. Our personal data ecosystems are incredibly complex and largely invisible. For example, it is easy to allow a handful of messaging and social media apps to access your contact list. Before you know it, you have created a complex and unmanageable network of connections that silently sync and propagate your addresses and phone numbers across the Internet. And there are deeper layers which are not even slightly visible to users: networks of data brokers, advertisers and digital cookie companies exchange user identifiers, activity data and personal information about you while you browse or use apps (Pidoux et al., 2022). The ability to build up a meaningful picture of your personal data ecosystem is completely absent [4.3.4] or severely limited. People remain ‘in the dark’, leading to fear (Bowyer et al., 2018), overload [2.2.4] and resignation [5.4.4]. Managing one’s personal data ecosystem is an overwhelming, unmanageable task that even personal data experts are not fully able to get a handle on. We do not feel ‘in control’ [Teevan (2001); 2.2.2]. The ability to provide a user with ecosystem transparency is hindered by the complexity and multiplicity of data relationships they have been encouraged to set up. People lack tools to provide a meaningful, or indeed any, view of those relationships. In both Case Study contexts, we saw that no one individual or organisation has the ability to see the whole of a user’s data ecosystem [4.3.4; Cornford, Baines and Wilson (2013)]. There is little commercial motive to try and solve this problem, as each provider focuses on their own apps, websites and services. Making one’s ecosystem visible, transparent and understandable is therefore an essential objective for better HDR, as Insight 4 shows.

8.3.2 A Lack of Metadata

As we start to consider what the data is about, new possibilities are unlocked. A PDS-type system could built that is not only a repository of personal data, but (using proxy representations), a collection of ecosystem information and contextually-situated life information too. This could include information about relationships with data holders or other entities. Builders of such a system would face a further challenge—a lack of metadata [2.2.2]. Typically, most data on our hard drives lacks context about its origin, and how it relates to the individual in a holistic life/ecosystem sense. Where data access rights are executed (or data is personally shared [4.3.2]), the attention is on the data itself: what it says. But as Case Study Two showed, some of the most desired information was not the data itself, but handling information and inferences—information that can only come from metadata, which was rarely forthcoming [Table 5.3]. Metadata could include many facets that could be quantified and recorded, as illustrated in Figure 8.5, which I created at BBC R&D:

These facets can be mapped back to the 5 W’s that collectively make up the user’s context [Abowd and Mynatt (2000); 2.2.2]. Many of these facets are not explicitly recorded today, or would take significant work to capture. Nonetheless, this exploration shows how data can be better contextualised, supporting contextual and associative approaches [2.2.2]. This leads to Insight 5:

Data must carry with it the details of why it exists, how it came to be, and what has happened to it since its inception.

Paying attention to ecosystem information, metadata and provenance, facilitates a new space that, at the time of writing in 2022, almost no-one is building for. For people to manage their digital world, they need a map. This is the first step on the road to giving individuals oversight of their personal data ecosystem.

8.4.1 Hegemony through Data Holding

It is in the pursuit of oversight [6.2.2] and involvement [6.2.3] that the impact of the power imbalance [2.1.2] becomes most clear; unlike the other HDR objectives, individuals cannot act to claim ecosystem negotiability for themselves. Negotiability means having the power to act, and in the context of systems and interfaces owned and designed by service providers, that power can only be given. The hegemony of data holders is therefore is the greatest obstacle to this objective, so it is vital to examine the nature of that power. Where does it come from?

A helpful analogy for the relationship between provider and user can be seen in the design of Jeremy Bentham’s panopticon (Bentham and Bozovic, 1791), a real-world version of which is pictured in Figure 8.6. The panopticon is an 18th century prison architecture that elevates the power of the (hidden) prison guards to observe all the prisoners easily at any time while removing prisoners’ privacy and providing them no ability to observe those in power. As in Orwell’s Nineteen Eighty-Four, individuals are unable to know when they are being watched (in this case, because the guards are hidden from view by one-way screens). This enforces compliance. Structuralist philosopher Foucault interpreted the panopticon as a political design, recognising that human environments can be configured to influence or regulate behaviour, in order to defend the power of the ruling class (Foucault, 1975). Such designs embody his four principles:

• Pervasive Power: the guards see everything all the prisoners do, all the time
• Obscure Power: the guards can see into any cell at any time, but the prisoners can’t know when, how or why they are being observed
• Direct Violence Made Structural: the structure motivates the prisoners to self-regulate their behaviour without being coerced (through beating or punishment)
• Structural Violence Made Profitable: having been made compliant by the structure, the prisoners can be put to work for the benefit of those in power, as it is the only option available to them.

We can see at least three of these traits in modern Internet platforms such as Facebook today. These platforms monitor users’ behaviour (pervasive power) without their knowledge and without accountability (obscure power). Interfaces are designed to offer only those actions that benefit the platforms (for example, clicking ads, sharing content or spending more time on site (structural violence made profitable). This has happened through the processes of platformisation and infrastructurisation (Helmond, 2015; Plantin et al., 2018), which have supplanted the Web 2.0-era promise of a free, open Internet that could have been more empowering to individuals.

Through the control of data and of interface design—the only channels through which they can be observed—service providers and platforms assert a structural power over the digital landscape. Just as the design of the panopticon regulates the behaviour of the prisoners, so the configuration of platforms, apps and service interfaces we use regulate and limit our behaviour as users. As Lessig wrote, ‘code is law.’ (Lessig, 2000). This infrastructural power is explained further in [Insight 6] below.

Structural power is not the only form of power which modern-day data-centric service providers hold. Jasperson et al.’s extensive review of types of power in the context of technology organisations (Jasperson et al., 2002) identifies 23 different power paradigms, of which at least 13 can be, and are, asserted by data-centric organisations today:

• authority: ownership of technology or infrastructure (for example of websites, servers and code)
• resource control: controlling the flow of resources (in this case of information/data)
• systems/structural power structural manipulation of others (as detailed above)
• rational power: controlling decision-making processes
• disciplinary power: using an influential position to affect others’ mental models (for example, positioning location tracking as theft resilience
• zero sum power: winning a battle for ownership/resource control at the other party’s expense (e.g. losing control of your sacrificed data)
• behavioural influence: persuading others to carry out the desired behaviour (e.g. restricting features to motivate subscription payments, or promoting certain content or actions)
• interpretative influence: determining how reality is externally represented (e.g. Facebook determining the way in which your social network is represented to you)
• network centrality: becoming an indispensable hub of a wider ecosystem (for example, Facebook/Google dominance in online ad-brokering)
• processual power: changing processes for competitive advantage (for example, platforms offering preferential APIs or rates to compliant partners)
• socially shaped power: influencing a wide audience to settle upon a preferred interpretation (e.g. using dominant market position to dominate debates e.g. about privacy norms)
• interpretive power: creating the internal representations of reality within an organisation (for example, presenting unpopular attitudes to data privacy to staff as normal/acceptable/beneficial for business)

• technical ability,
• organisational ability, and
• the acquisition of data about individuals and populations.

1. Collect & Interpret Data to Acquire Knowledge: Data and signals are collected from individuals and interpreted in order to infer their intents and interests. For example, Google collects raw GPS and wi-fi hotspot data from mobile phones, which it then statistically analyses to infer which shops or venues you visited and what forms of transport you used, increasing Google’s knowledge about individuals and populations.
2. Present Content and Configure Structures to Influence Individual Behaviour: Knowledge of individual intents and interests is exploited within user interfaces to influence desired individual actions. For example, Facebook or presents a user with a product relevant to their interests, which they are motivated to click upon, generating ad revenue. Another example would be Twitter manipulating the content of the user’s feed to show more tweets from conversation topics where they can show promoted tweets, increasing ad revenue.
3. Configure Structures to Improve Knowledge Acquisition: A provider uses its dominant position to force other organisations to improve the provider’s ability to acquire knowledge. For example, Google provides free analytics tools to web developers, but requires the end users of those client websites to supply visitor data back to Google, increasing their ability to acquire knowledge about individuals and populations.
4. Configure Structures to Disadvantage Others: Certain providers (typically of operating systems or popular devices) can configure the structural relationships between other parties. For example, a smartphone manufacturer could limit data exchange between other apps, while still extensively collecting data signals themselves, such as when Google was found to be collecting call history from Android’s dialer app.

8.4.2 The Active Diminishing of User Agency

The second major obstacle to ecosystem negotiability is that platformisation and power exertion are not a one-off transition, but rather an ongoing process. Today’s platforms exhibit a continuing trend of actively diminishing individuals’ agency, especially in the last decade. When software was sold in a box, manufacturers competed based upon which product would let the user take home the greatest range of features and capabilities. New releases with new features drove new product sales. But in the cloud computing era, a smaller set of core features done well is sufficient to guarantee an ongoing subscription revenue from a user. Cost savings in development and support costs can be made by reducing feature sets. Constrained, compliant users are easier to manage. The relentless pursuit of increased profits and further cost saving sees products lose, not gain, features. Interfaces are reshaped to serve businesses’ interests first and foremost. Providers’ focus on making user behaviours constrained, predictable and profitable, more than meeting their needs or providing maximal value [2.3.5]. Plantin et al. describe the particular harmful influence on the ecosystem of Facebook’s power exertions:

“Facebook is a formidable force in a profit-motivated platformisation which is beginning to eat away at the Open Web. This entails moving away from published URIs and open HTTP transactions in favour of closed apps that undertake hidden transactions with Facebook through a Facebook-controlled API.”—Plantin et al. (2018)

Here are just a few examples of the ways in which users’ agency has been, and continues to be, diminished:

• Facebook closed their RSS feeds, and later parts of their APIs, meaning that users could no longer consume their friends’ posts in any other environment than the ad-filled and manipulated Facebook main feed. Later, they eliminated feeds of friends’ posts and favourite pages, removing users’ ability to compartmentalise their content viewing to certain friends groups. The ‘Friends’ page on Facebook currently shows a list of recommended new friends. To access your current friend list requires an extra click. Encouraging users to grow their networks is prioritised over user convenience.
• Twitter closed the parts of its APIs that allowed real-time notifications and access to one’s home feed, killing off primary functionality for a healthy ecosystem of third-party Twitter clients that increased user choice (Newton, 2018). TweetDeck, a major third-party Twitter client was acquired, and later shut down, as was Twitter’s own desktop client. Eventually, the only option left to users was to use the web interface. (Gayomali, 2015; Hatmaker, 2018; Siegal, 2022)
• Apple has been diminishing users’ agency for a long time. Users cannot open up iPhones even to change the battery without invalidating their warranty. Apple have removed disk drives, headphone ports, SD card slots and other ports. Certain parts of the hard drive on macOS devices are now read-only and non-writeable by users.
• Facebook recently announced they will no longer store users’ historical location data (though they will still use location information) (Pegoraro, 2022). This means users will lose the capability to access historical location records. But this makes it harder for users to see how their location data will be used in future, as there will be no historical log to examine. Data-centric companies can change their practices to limit agency and reduce accountability.
• Online news and discussion site Reddit has removed content access for non-logged in users, and uses deceptive techniques to present advertisements that look like posts from users, and to discourage users from appearing offline. These patterns have been described as disrespectful design (Regoje, 2021).
• In an example from the public sector, through my work on the SILVER project [3.4.1] just prior to the introduction of the GDPR in 2018, I heard whispers in at least one local authority of plans to ‘shift from getting data collection consent from supported families towards simply informing them of our practices’ (in other words, removing their choice). The instinct to further organisational interests over those of the individual appears not to be limited to commercial data holders.
• In a similar vein, TikTok recently announced that it would rely on legitimate interest rather than consent when it comes to using users’ activity data to personalise the app experience. This removing users’ ability to withdraw consent to such use. This plan has subsequently been paused after warnings that this might breach GDPR (Lomas, 2022).

Unchecked, trends to reduce users’ agency and further providers’ interests at the expense of human autonomy are likely to continue. Today’s data-centric systems suffer from a lack of consideration to individual welfare. Data centricity encourages neglect of the human end user perspective, creating potential for harm:

“There are certain things you do not in good conscience do to humans. To data, you can do whatever you like.”—Sonnad (2022)

The trend to diminish users’ agency is needs explicit targeting if data interfaces are to become more free-flowing (Bowyer, 2018), and if ecosystem negotiability is to be realised. Somehow, the trend needs to be halted, before it can be reversed. The TikTok example suggests this can only be achieved through regulatory changes.

8.4.3 The Intractable Data Self

The third obstacle to ecosystem negotiability is the intractable data self. Data about individuals serves as their proxy [[Bowyer et al. (2018); 5.4.4]. This is their data self [4.4.1]. If it is incomplete, inaccurate or unfair—highly likely given the difficulties of representing people in data (Martin, 2007; Cornford, Baines and Wilson, 2013)—which can cause harm (Bowyer et al., 2018; Crossley, 2022). Yet currently, although some legal rights to data correction exist (Information Commissioner’s Office, 2018), people cannot modify or assert control over this most important version of themselves—the version of them that exists in data. Even when data can be seen, people lack the ability to exert influence over their data self [5.5.2; Cornford, Baines and Wilson (2013)]. To address this obstacle, HDR reformers should explore giving people a role in the curation of their data self [4.4.3; 5.5.2] and 6.3].

To date, research and innovation on ecosystem negotiability has been very limited. It is easier to find business models and research funding for narrow and well-defined contexts. Without a business motive, only non-profit socially-focussed research organisations such as BBC R&D and Sitra have found themselves well-equipped to explore this problem space. Nonetheless, there is an urgent societal need. People need to reclaim their data selves, and be given control over their digital lives at the broadest level.

8.5.1 A Lack of Individual Demand

Customers are not demanding HDR capabilities in their lives, and, all but the most socially-responsible businesses do not see value in an approach that runs so contrary to current business models, based as they are on data accumulation and the constraining of customer experiences.

Data is overwhelming, complex, and ‘sounds boring’. Engaging with your personal data economy to any degree more than that of passive consumer is hard work. People routinely accept data sacrifice, click through T&Cs and cookie banners and are unwilling (or in some cases lack sufficient technical literacy, comprehension or skill) to do the work of asserting control over their digital lives. There is not a clear demand for holistic digital life management and control. Research in this this and at Cornmarket suggests that even if human-centric information systems and more inclusive service interaction practices emerged, people would not be inclined to use them in great numbers. It could seem like hard work or not worthwhile. This obstacle that affects all HDR improvement approaches. Indeed is why many companies in the emergent PDE economy [2.3.4] struggle to find a business model. There are clear benefits, but better HDR does not appear to something a mainstream audience will pay for. This should not deter disruptive innovation nor diminish the potential value for such tools. As automobile pioneer Henry Ford famously said, “If I had asked people what they wanted, they would have said faster horses.” Nonetheless, it is a clear overarching obstacle, which Insight 7 attempts to address.

8.5.2 Closed, Insular and Introspective Practices

The kind of life-spanning, unifying interfaces described in the insight above are nothing like the interfaces that are built today, as they span across different providers’ data and services. This highlights the secondary obstacle that all HDR system builders will face, whichever objective they wish to target: closed, self-interested organisations with a lack of interoperability. Building an HDR system will necessarily involve connecting to systems of different providers that have different touchpoints into an individual’s life and world. Yet most companies act in closed, introspective and non-cooperative ways to further their own interest. Companies like Apple, Amazon, Microsoft, Facebook and Google (the so-called ‘big five’) build proprietary, incompatible silos or ‘walled gardens’—sub-Internets that pretend that the alternatives do not even exist, in order to encourage a flow of money and attention to their own products and services.

Commercial motives encourage them to get users to spend time in their own proprietary spaces (so that resultant ad revenue can be captured) and in order to maintain subscription revenues it is in providers’ interests to make it hard for individuals to leave or switch providers. In effect, providers build for a world that does not exist, where every individual is imagined to only interact with that single company’s interfaces. I would argue, for example, that Google’s venture into social networking with Google+ did not succeed because it failed to build for a reality where most people and their friends were already on Facebook.

8.5.3 A Lack of Organisational Investment in HDR

But one can understand their motives; there is little incentive to open up the ecosystem when the free flow of information and of users might result in loss of income for the company in question. Users with negotiability would be more able to leave. And this also encourages keeping users in the dark [5.4.2]. The less agency and negotiability that users have, the more freedom the provider has to do exactly what they want with their data. In this context, users are ‘pathetic dots’ (Lessig, 2000) or ‘docile bodies’ (Foucault, 1975).

The tendency of organisations to work in closed, introspective practices and to be resistant to opening up data or services is not solely motivated by commercial reasons: the public sector has a vastly complex, closed and fragmented ecosystem [Pollock (2011); Copeland (2015); 4.1.2]. Efforts to build a system to share health data with support workers for the SILVER project [3.4.1] proved hugely challenging. Sometimes the challenge was a more technical one - incompatible data formats that are hard to reconcile, or data being stored in legacy systems with no public API that would allow programmatic access to that data, or issues around licensing. Data sharing agreements must be established, especially in the public sector which is by its nature more liable to scrutiny and accountability. But more than these technical or procedural issues there was resistance to change in data processes and an unwillingness to share data between agencies, often motivated by a fear of legal repercussions. Data-centrism encourages insular thinking: it encourages organisations to codify the world into their own systems, processes and formats for their own use.

8.5.4 A Lack of Interoperability

And yet, for effective HDR, data needs to be separable from services. The more users data is tightly coupled to specific services, the less agency users have and the harder it is to build life-centric systems. On BBC R&D’s Cornmarket project, attempts to build an interface for users to import data from multiple popular Internet services proved to be a hugely complicated endeavour, requiring access to many different APIs or manual exports and imports of data by users. There needs to be greater interoperability and greater establishment and adoption of standard formats for exchanging human information (as distinct from establishing standards for data or service-specific APIs). As mentioned above, platformisation breaks the Open Web (Plantin et al., 2018). To overcome this, companies must be persuaded that human-centric thinking, interoperability and transparency has not just social benefits, but business benefits too.

8.5.5 Insufficient Machine Understanding of Human Information

But at an abstract level the technical obstacle, the problem is one that has always faced the tech industry, which is that there often is no universally agreed way to represent important concepts - in this case human-centric information concepts such as events, social media posts, website visits, location history information, app activity, etc. And any entity that does create a standard then faces the challenge of trying to persuade others that their standard is the best one to use. In general, standards work best when established by non-commercial industrial standards bodies (for example the World Wide Web Consortium (W3C) or International Organisation for Standardization (ISO) and then mandated through policy such as European Union law. Such standards much be established with input from industry experts.

Machine learning technologies and Artificial Intelligence have pushed machine understanding of human words, images and content to impressive levels in recent years and such technologies can certainly be helpful, but in fact at the core what we are talking about here is something much simpler than AI; It is simply about labelling datapoints in as many different ways as possible so that those datapoints can be associatively retrieved from many different angles, and providing humans with ways to amend incorrect labels and to reclassify data or apply new semantic associations. Issues of interoperability for PDS systems are being actively explored and developed in the ‘Solid’ community (Bansal, 2018; Berners-Lee, 2022) in pursuit of a decentralised web (Verborgh, 2017).

Such approaches are in their infancy, and have not yet been adopted extensively in commercial settings. Even after addressing the obstacles of end-user buy-in and the technical complexities of building human-centric systems, data-driven corporations, motivated as they are by profit and business success (and smaller online organisations too) need to be persuaded of the business value of transparency, interoperability and human-centricity. This is explored further in 9.5.

In summary, whichever of the above four HDR objectives are targeted, all HDR reformers involved in building HDR systems must:

1. create, adopt and co-ordinate around new standards for human information storage and management
2. invest in systems that elevate computers from data-processing machines to human-information-processing machines, and
3. make a persuasive case to both businesses and individuals that the new approach offers tangible, previously unavailable value.